Depth Based Image Registration via 3D Geometric Segmentation

Image registration is a fundamental task in computer vision and it significantly contributes to high-level computer vision and benefits numerous practical applications. Although many image registration techniques have been proposed in the past, there is still a need for further research because many issues such as the parallax problem remain to be solved. The traditional image registration algorithms suffer from the parallax problem due to their underling assumption that the scene can be regarded approximately planar which is not satisfied when large depth variations exist in the images with high-rise objects. To address the parallax problem, we present a new strategy for 2D image registration by leveraging the depth information from 3D image reconstruction. The novel idea is to recover the depth in the image region with high-rise objects to build accurate transform function for image registration. We use a geometric segmentation algorithm to partition 3D point cloud to multiple geometric structures and at the same time, estimate the parameters of each geometric structure. Experimental results show that our proposed method is able to mitigate the parallax problem and achieve better performance than the existing image registration scheme.